Method for generating test patterns for detecting and quantifying losses in video equipment

ABSTRACT

A method for detecting and quantifying losses in at least one of video and audio equipment, comprising the steps of generating the test pattern; processing the test pattern through video equipment; and displaying the processed test pattern to a display to a viewer. In one embodiment, the test pattern is indicative of video compression losses due to quantization. In another embodiment, the test pattern is indicative of a color transformation mismatch after encoding/decoding with incompatible video transmission standards. In a third embodiment, the test pattern is iso-luminant after a color transformation between a first and second video transmission standard. In a fourth embodiment, the test pattern is indicative of lipsync error.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/132,059 filed Jun. 3, 2008, which further claims the benefit of U.S.Provisional Patent Application No. 60/941,771 filed Jun. 4, 2007, U.S.Provisional Patent Application No. 60/941,773 filed Jun. 4, 2007, U.S.Provisional Patent Application No. 60/941,776 filed Jun. 4, 2007, andU.S. Provisional Patent Application No. 60/978,567 filed Oct. 9, 2007,the disclosures of which are incorporated herein by reference in theirentireties.

FIELD OF THE INVENTION

The present invention relates generally to video transmission, and moreparticularly to a method for generating test patterns for visually andnumerically indicating losses in at least one of video and audioequipment.

BACKGROUND OF THE INVENTION

Video (and audio) has moved into the digital age, wherein still picturesand video are now imaged by CCD solid state devices instead of “analog”chemical films, and processed by microprocessors. One side effect of thedigitization process is that very large amounts of data are produced,which may be larger than the memory capacity of the digital imagingdevices. Lossless and lossy compression algorithms have been developedto reduce these large amounts of data to a manageable size.Unfortunately, completely lossless compression reduces the size of filesby only about 50% or less. Much higher compression ratios are needed,which require lossy compression methods. It is the goal of such methodsthat the resulting losses would not be noticeable to the human visualsystem. While several standards have been developed with these goals inmind, some unintended artifacts may be produced in the resulting images,which may be visually noticeable. Many of today's video encoders attemptto reduce or remove distorting elements that would likely be missed by ahuman viewer, but this is a matter of subjective judgment.

It is desirable to quantify the degree of distortions. Compressiondistortion is often quantified using a “before and after” techniquewherein a computer compares frames before and after encoding. Thisrequires access to the original frames. Others have created“single-ended, reference free” techniques, but these techniques cannotdiscern small distortions, which renders them unsuitable forprofessional work. These techniques are also sensitive to the underlyingvideo content. Test signals such as “color bars” and “multiburst” havebeen in use for many years, but were designed to test degradations ofanalog signals, and are thus unsuitable for testing digital compressionsystems.

Another class of distortions involves “transformations” and “reversetransformations” of colors in images using incompatible transmissionstandards. For example, each colored pixel in a video image isrepresented as a set of three values corresponding to red, green, andblue (RGB). It is common practice to perform a reversible matrixtransformation on a first set of red, green, and blue values to obtain adifferent set of values, e.g., Y′CbCr prior to transmitting or encodingthe Y′CbCr values, then transforming the Y′CbCr values back to RGBvalues. One advantage of performing this type of transformation is thatthe resolution of the two color difference arrays, Cb and Cr, can bereduced as a form of compression. There exists more than one reversibletransform for this purpose. For many years the “Rec.601” standard (anabbreviation for ITU-R Recommendation BT.601) was used for standarddefinition video, but with the introduction of high definitiontelevision, a new transform was defined for high definition video, knownin the art as “Rec.709” (an abbreviation for ITU-R RecommendationBT.709). Unfortunately, these two standards are incompatible, i.e.,encoding with one standard while decoding with the other standardintroduces color errors. It would be desirable detect when such atransform mismatch has occurred.

Another issue related to reversing RGB to Y′CbCr is the creation of testpatterns in the Y′CbCr domain for analyzing the performance of equipmentthat processes video in that domain. Further, it is desirable to testthe Cb and Cr channels independently from the Y′ channel. For example, atest pattern may be synthesized so that the Y′ “luma” component has aconstant value, while the Cb and Cr components are varied in some waythat will challenge the equipment under test. Such a pattern may betermed an “iso-luma” pattern. An example is shown in FIG. 1, which isfrom a Snell and Wilcox test pattern. FIG. 1 shows the alternating green4 and purple 6 bands typical of a signal applied equally to both Cb andCr channels.

Unfortunately, if an “iso-luma” pattern was transcoded from, say, the“Rec.601” domain to the “Rec.709” domain (or vice versa), the lumacomponent would not likely remain constant, and the iso-luma property ofthe pattern would be lost in the transcoding operation. It would bedesirable for a test pattern to remain iso-luminant despite having beentranscoded from one domain to another domain.

Since the introduction of “talkies” (movies with sound), synchronizingthe image sequence and the sound has been an issue. The film soundpickup head was located beyond the lower loop, and a loop that was toosmall would result in the sound being played too early. The error ismost easily detected in scenes where people are talking, because theirlips would be out of synchronization with the sound. Thus, a timingoffset between video and audio streams, called “lipsync error” wouldresult.

Analog television systems did not suffer much from lipsync errors untilthe advent of video processing techniques that used one or more framesof delay, such as a frame synchronizer or a digital video effects unit(DVE). As the cost of electronic storage has dropped, more and moreframes of delay are included in the processing path of the video. Sinceaudio is usually handled in a separate signal path during production,the equipment that introduced the video delay usually has no means ofcorrecting for the video delay in the audio path. Video compressionsystems use 0.5 to 1.0 second or more of buffering in the process ofencoding and decoding video. Although compression standards are quiteclear about how to avoid lipsync errors, mistakes creep in. Lipsyncerrors can accumulate as the video and audio progress through anequipment chain.

It is desirable to be able to quantify the lipsync error. Further, it isdesirable to quantify this error simply by observing a video/audio testsequence, without resorting to companion equipment.

One method in the prior art for indicating lipsync error is a testsequence wherein a rotating “clock hand” passes through the verticalposition at the same time an audio event, such as a tone or click, isheard. This pattern helps determine that lipsync error has occurred, butdoes not quantify the error. Other products have been developed whichinclude a frame synchronizer with an integrated audio delay, and haveincluded a built-in test sequence to facilitate adjustment of the delay.Other proposed test sequences include a video element that flashes atthe time of a tone; some equipment manufacturers have proposed companiontest equipment (a light sensor and a microphone) to facilitate errormeasurement. But such companion equipment is, at times, inconvenient anddepends primarily on a trial-and-error approach to correct lipsyncerror.

Accordingly, what would be desirable, but has not yet been provided, isa method for generating more numerically and visually discriminativetest patterns for effectively and automatically quantifying losses in atleast one of video and audio equipment.

SUMMARY OF THE INVENTION

The above-described problems are addressed and a technical solution isachieved in the art by providing a method for detecting and quantifyinglosses in at least one of video and audio equipment, comprising thesteps of generating a test pattern; processing the test pattern throughvideo equipment; and displaying the processed test pattern to a displayto a viewer.

In one embodiment, the test pattern is indicative of video compressionlosses due to quantization, the test pattern including a periodic signalof at least one frequency distributed over a plurality of levels ofamplitude in at least one of space and time such that, when a level ofamplitude falls below a predetermined level, there is a cessation ofcontrast from a background pattern, the cessation of contrast beingindicative of the step size of quantization. The periodic signal can beindicative of a plurality of frequencies and can be distributed overspace. The test pattern can comprise a set of concentric rings of atwo-dimensional sinusoidal pattern. The frequency of the test patterncan increase from the center of the test pattern to the Nyquist limit ofthe video sampling rate at the edges, the test pattern being modifiablein both amplitude and frequency. The amplitude of the test pattern candrop with angle. The step size of quantization can be indicated as aclock position.

In another embodiment, the test pattern comprises at least two colors,wherein when a color transformation mismatch occurs in the system undertest due to processing with incompatible video transmission standards,the resulting pattern displays a reduction in contrast between the atleast two colors as a result of clipping. When one color is about 100%of its range and the other color is set to about the reciprocal of thegain increase to a color that occurs when a transformation mismatch ispresent in the video equipment. The at least two colors can berepresented as a shape within a shape, or can be represented as adjacentshapes that flash in time. The present invention can be used fordetecting a color mismatch transformation occurring between equipmentbeing encoded or decoded with the Rec. 709 and Rec. 601 standards.

In a third embodiment, the test pattern has luma and chroma parts, theluma part of which remains substantially unchanged after a colortransformation between a first and second video transmission standard,the test pattern being generated from a locus of color values that isthe intersection between a first plane representing a first weighted sumof color values in a color space and a second plane representing asecond weighted sum of color values in the color space, the first sumbeing weighted based on the first video transmission standard and thesecond sum being weighted based on the second video transmissionstandard. The test pattern can be based on finding the intersection ofthe planes formed by the equationY=(R601*R)+(G601*G)+(B601*B) and Y=(R709*R)+(G709*G)+(B709*B)where R601, G601, B601 represent constant weights in a range between 0and 1 for the colors red, green, and blue according to the Rec. 601transmission standard, respectively, R709, G709, and B709 representconstant weights in a range between 0 and 1 for the colors red, green,and blue according to the Rec. 709 transmission standard, respectively,Y is a predetermined constant value of luma, and R, G, and B are red,green, and blue color space variables, respectively.

In a fourth embodiment, the test pattern is indicative of lipsync error,the test pattern comprising at least two distinct visual features thatvary in time, and a corresponding audio sequence containing one or morechanges in the sound that correspond to a change in one or more of theat least two distinct visual features. The at least two distinct visualfeatures vary in time by means of a change in at least one ofbrightness, color, size, and shape. The at least two distinct videofeatures for indicating lipsync error can be arranged in a linearpattern. The at least two distinct visual features that vary in time canbe a plurality of video tick marks of one color, each tick mark changingin brightness for a predetermined amount of time in sequence, the audiosequence being an audio “tick” that, when video tick marks change inbrightness, the audio tick mark is used by a viewer to gauge which videotick mark that changes in brightness is nearest in time to the audiotick mark, the nearest changing video tick mark indicating the number offrames of lipsync error.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more readily understood from the detaileddescription of exemplary embodiments presented below considered inconjunction with the attached drawings, of which:

FIG. 1 is a diagram depicting an “iso-luminant” pattern in the priorart;

FIG. 2 is a block diagram of a hardware architecture of a system forproducing test patterns, constructed in accordance with an embodiment ofthe present invention;

FIG. 3 is a diagram depicting a video test pattern for detecting andquantifying losses in one or more video images, constructed inaccordance with a method of the present invention and the apparatus ofFIG. 2;

FIG. 4 is a diagram depicting a video test pattern similar to the testpattern of FIG. 3, wherein the sine wave patterns stops abruptly at the8 o'clock position, indicating 8-bit performance from 0 to the Nyquistlimit;

FIG. 5 is a diagram depicting a video test pattern similar to the testpattern of FIG. 3 as would be produced after processing with a resultingpattern for JPEG compression using Adobe Photoshop™ with a “quality”quality setting of “9”;

FIG. 6 is a block diagram of an apparatus for generating the testpatterns of FIGS. 3-5, constructed in accordance with an embodiment ofthe present invention;

FIG. 7A is a diagram depicting a video test pattern used to detect aforward color transformation mismatch for the Rec. 709 and Rec.601 videocompression standards;

FIG. 7B is a diagram depicting a video test pattern used to detect areverse color transformation mismatch for the Rec. 709 and Rec.601 videocompression standards;

FIG. 8 is a block diagram of an apparatus for generating one “saturated”color channel of the test patterns of FIGS. 7A and 7B;

FIG. 9 is a diagram depicting an iso-lumanant test pattern, constructedin accordance with an embodiment of the present invention;

FIG. 10 is a block diagram of an apparatus for generating aniso-luminant test pattern for transformations between the Rec.601standard and the Rec.709 standard, constructed in accordance with anembodiment of the present invention;

FIG. 11 is a diagram depicting a video test pattern for indicatinglipsync error, constructed in accordance with an embodiment of thepresent invention;

FIG. 12 shows portions of the video test pattern of FIG. 11 in which a“video tick” moves from left to right; and

FIG. 13 is a block diagram of an apparatus generating a video/audio“test pattern” for detecting and evaluating lipsync error, constructedin accordance with an embodiment of the present invention.

It is to be understood that the attached drawings are for purposes ofillustrating the concepts of the invention and may not be to scale.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to FIG. 2, a test pattern generation system for detectingand quantifying losses in at least one of video and audio equipment isdepicted, generally indicated at 10. By way of a non-limiting example,the system 10 includes a computing platform 16. The computing platform16 may include a personal computer, work-station (e.g., a Pentium-M 1.8GHz PC-104 or higher), or an embedded system comprising one or moresingle or multi-core processors 18 which includes a bus system 20 whichconnects the one or more processors 18 with the computer-readable medium24. The computer readable medium 24 can also be used for storing theinstructions of the system 10 to be executed by the one or moreprocessors 18, including an operating system, such as the Windows or theLinux operating system, and at least one of video and audio datacorresponding to test patterns to be discussed hereinbelow. The one ormore processors 18 generate one or more video and/or audio test patternsfrom the computer-readable medium 24, thereby functioning as a testpattern generator. The computer readable medium 24 can include acombination of volatile memory, such as RAM memory, and non-volatilememory, such as flash memory, optical disk(s), and/or hard disk(s). Asynchronization timing reference (Genlock) 25 can be supplied to thesystem 10 for synchronizing the system 10 to other equipment. Aprocessed video data stream 26 can be stored temporarily in the computerreadable medium 24 or fed in real time to the equipment under test 27which in turn feeds a test pattern to a monitor 28 or projects the testpattern through a projector 30 on a projection screen 32. The testpattern can be one or more or a composite of all of the embodiments oftest patterns to be described hereinbelow. The test pattern can providevideo and optional audio indications of the degree of error introducedby the equipment under test 27 which may have compression or some othercause of distortion. The video/audio test patterns can provide directionindication of distortion and its quantification, or provide video and/oraudio cues to a viewer.

In other embodiments of the system 10, the one or more processors 18 canbe replaced by a digital signal processor, a field programmable gatearray, and application-specific integrated circuit, or customanalog/digital electrical circuitry.

FIG. 3 displays a video test pattern 34 for detecting and quantifyinglosses in video equipment, constructed in accordance with a method ofthe present invention. The test pattern 34 shows a visual representationof compression losses, e.g. due to quantization, spatially in twodimensions. The test pattern includes a set of concentric rings 36,preferably a two-dimensional sinusoidal pattern, whose frequencyincreases from the center 38 of the test pattern 34 to as much as theNyquist limit of the video sampling rate at the edges 40 of the testpattern 34.

The test pattern 34 can be modified in both amplitude and frequency. Thefrequency at the edges 40 is the Nyquist limit, and in a preferredembodiment, the frequency increases smoothly with equally-spaceddoublings starting at the center 38 at some non-zero value, so that eachoctave represents an equal distance along a radius (e.g., 100 lines, 200lines, 400 lines per unit height, as shown). Slight changes can be madein the background level for each octave so that a viewer of the testpattern can estimate frequencies, which can be labeled in units of linesper picture height. In another embodiment, the frequency can droptowards the center, i.e., the reverse of the preferred embodiment.

FIG. 4 demonstrates the effects of distortions due to compression whichreveals itself as bit-level of performance or alternatively expressed asthe step size of quantization. The amplitude of the test pattern 34 isnot constant but drops with angle. In the present embodiment, thebit-level of performance is indicated as a clock position 44. If theviewer can see the two-dimensional sine wave at the 6 o'clock position,then the system under test has at least 6-bit performance (64 levels).If the viewer sees the sine wave at the 8 o'clock position, the systemunder test can resolve 8 bits, or 256 levels. As the encoder qualityvaries, so does the angle where the sine wave vanishes. In the patternshown in FIG. 4, the sine wave patterns stops abruptly at the 8 o'clockposition, indicating 8-bit performance from 0 to the Nyquist limit.

FIG. 5 shows a resulting pattern for JPEG compression as would beproduced using Adobe Photoshop™. Adobe Photoshop™ has a “quality”parameter that can be varied from 1 to 12. FIG. 5 shows acontrast-enhanced output pattern for a quality setting of “9.” Thequality of compression can be interpreted as 8 bits out to 200 lines,but tapering off to 7 bits at 400 lines.

The test pattern of FIGS. 3-5 is not strictly limited to a spatialdistribution of a two dimensional sinusoid that varies in amplitude andfrequency. In the more general case, the test pattern can include aperiodic signal of at least one frequency distributed over a pluralityof levels of amplitude in at least one of space and time such that, whenamplitude falls below a predetermined level, there is a cessation ofcontrast from a background pattern, the cessation of contrast beingindicative of the step size of quantization.

FIG. 6 is a block diagram of an apparatus 46 that can be implemented bythe system 10 of FIG. 2 for generating the test patterns of FIGS. 3-5,constructed in accordance with an embodiment of the present invention.The apparatus 46 includes a plurality of parameter generators 48, 50,52, 54 which feed their outputs to a function generator 56 to generate atest pattern 58 based on the function is A cos(f+p)+c, where A, f, p,and c are the intermediate outputs of the parameter generators 48, 50,52, 54, and stand for amplitude A, frequency f, phase p, and offset c,respectively, to be described hereinbelow. The inputs to the apparatus46 are x, y, and t. The x and y inputs indicate the spatial position ofa given pixel, and the t input indicates the temporal position, e.g.,frame number. The four parameters A, f, p, and c are then used by thefunction generator 56 to create a brightness value for each pixel.

The amplitude generator 48 converts the x and y values to a vectorangle, which then outputs a value that is 1.0 at the “noon” position anddrops exponentially by half for each 1/12 of a full revolution, that is,for each ‘hour’ of the clock face. At the 1:00 position, the amplitudegenerator 48 outputs 0.5, at 2:00 it outputs 0.25, at 3:00 it outputs0.125, and so on. The output of the amplitude generator 48 may be asmooth function of the angle, or it may be stepped.

The frequency generator 50 converts the x and y values to a distancefrom the center of the test pattern 58 representative of a frequencyvalue. In the preferred embodiment, the frequency rises exponentially asthe distance from the center increases, for example, starting at 50lines per picture height (LPH) at the center, then 100 LPH at certaindistance d from the center, then 200 LPH at 2d, then 400 LPH at 3d, andso on. The scale factor d is chosen so that the frequency reaches theNyquist limit at the edge of the pattern.

The phase generator 52 creates its output in proportion to time t. Inthe preferred embodiment, p is independent of x and y, and t is theframe number. The output of the phase generator 52 is p=t/N*2*π, where Nis the number of frames in a repeat period. For example, if the framerate is 30 fps, and the repeat period is two seconds, N=2*30=60. Thiscauses the test pattern to appear to move, and this movement isimportant in detecting losses in motion-compensating compressionsystems.

The offset generator 54, in a preferred embodiment, is constant andindependent of x, y and t. The offset can be as large as one half of thefull range, e.g., 128 for an 8-bit system. The offset produces aconstant background on which to look for variations. Another benefit ofthe offset is that it allows the largest possible amplitude withoutclipping or limiting the amplitude of the test pattern. However, in agiven cycle of the output, all of the bits will be changing as thesignal moves above 128 (1000 0000) and then less than 128 (e.g. 127=01111111). Loss of bits by simple truncation would not result in a constantvalue, but alternate between 1000 0000x and 0111 1111x. Thus, the lossof, for example, the LSB would not be apparent, since the values wouldstill be changing, even though it had been alternating between 127 and128. If the two values are 128 and 129 (1000 0000 and 1000 0001) thenlosing the LSB results in a constant value: 1000 0000x. If the amplitudeis larger e.g. +/−4 levels, the offset must be such that only the bottomfew bits are changing. The offset should be one-half of the amplitude,and is thus a function of x and y. The offset generator output wouldthen be the same as the amplitude generator output followed by adivide-by-2 operation.

Additionally, the offset generator 54 may be used as a kind of “scale”indicator, allowing the viewer to distinguish the frequency octave. Theoctave nearest the center may be indicated by slightly increasing theoffset, the next octave by slightly decreasing the offset, and so one.In a similar way, amplitude bands may be indicated: the “pie slice”between “noon” and “one o'clock” reverses the offset used to indicatethe octave (increases become decreases and decreases become increases);the slice between 1:00 and 2:00 is not reversed; the slice between 2:00and 3:00 is reversed, and soon. Note that varying the offset with theamplitude reduces the maximum amplitude that is possible by at leasthalf, since the MSB does not change.

The frequency and amplitude bands are generally indicated by modifyingthe luminance offset, but this may produce compression artifacts thatare may be distracting to the observer. To overcome this problem, in apreferred embodiment, the zone plate information can be carried in theluminance (Y) channel, while the frequency and amplitude information iscarried in the color difference channel(s) U and/or V.

For displaying a test pattern that indicates distortions produced by acolor transformation mismatch, according to another embodiment, a signalwith two or more distinct color values is chosen such that, when a colortransformation mismatch occurs, the resulting pattern would show thatone or more of the two or more distinct color values would exceed thenormal 0% to 100% range of that color. In a video system that enforcesthe range to be between 0 and 100%, values for color exceeding thatrange would be modified (i.e., clipped), and the distinction between thetwo or more colors would be reduced or eliminated. This loss ofdistinction would be an indication to a viewer that a colortransformation mismatch has occurred.

FIGS. 7A and 7B show test patterns of the present method used to detecta forward color transformation mismatch and a reverse colortransformation mismatch, respectively, for Rec. 709 and Rec. 601. InFIG. 7A there is shown, at left, a color test pattern 60 beforeencoding/decoding comprising a pair of color coded rectangular boxes 62(red), 64 (green), representing two of the three RGB colors, each of theboxes 62, 64 having an inner box 66, 68, a pair of rectangular rings 70,72 being formed therebetween. In FIG. 7A there is also shown a Rec.709encoding block 74, a Rec. 601 decoding block 76, and, at right, a colortest pattern 78 after encoding/decoding comprising a pair of color codedrectangular boxes 80 (red), 82 (green), representing two of the threeRGB colors, each of the boxes 80, 82 having an inner box 84, 86, a pairof rectangular rings 88, 90 being formed therebetween. The presentinvention is not limited to square or rectangular test patterns, butwould work equally well with other shapes, such as circles, triangles,etc. The outer rectangular rings 70, 72 represent a specific color atits maximum value of 100% or 1.0000, while the inner boxes 66, 68display the current color of that color type of at least an image patchbefore encoding. In the example shown in FIG. 7A, the inner green box 68has a value of 0.85324. The video patch is then encoded with the Rec.709 encoding block 74, to produce YUV. The YUV encoded patch is thendecoded with the Rec. 601 decoding block 76. The boxes 80 (red), 82(green) of the test pattern 78 at right represent the results afterencoding/decoding. When encoded with Rec. 709 and then decoded with Rec.601, the corresponding green values become 1.172 and 1.000,respectively. After limiting to a range of 0 to 100%, the values in box68 and ring 74 become indistinguishable at values of 1.0000 and 1.0000.

The pair of green values in box 68 and ring 74 is suitable for theforward encoding/decoding steps, but is unsuitable for reverse encodingwith Rec. 601 and decoding with Rec. 709. Instead, the red boxes/ringsare used for this purpose. Referring now to FIG. 7B there is shown, atleft, a color test pattern 92 before encoding/decoding comprising a pairof color coded rectangular boxes 94 (red), 96 (green), representing twoof the three RGB colors, each of the boxes 94, 96 having an inner box98, 100, a pair of rectangular rings 102, 104 formed therebetween. InFIG. 7B there is also shown a Rec. 601 encoding block 106, a Rec. 709decoding block 108, and, at right, a color test pattern 110 afterencoding/decoding comprising a pair of color coded rectangular boxes 112(red), 114 (green), representing two of the three RGB colors, each ofthe boxes 112, 114 having an inner box 116, 118, a pair of rectangularrings 120, 122 formed therebetween. In the example shown in FIG. 7B, theinner red box 112 has a value of 1.000 and the outer red ring 120 has avalue of 1.000. The video patch is then encoded with the Rec. 601encoding block 106, to produce YUV. The YUV encoded patch is thendecoded with the Rec. 709 decoding block 108. The boxes 112 (red), 114(green) at right represent the results after encoding/decoding. Whenencoded with Rec. 601 and then decoded with Rec. 709, the correspondingred values become 1.086 and 1.000, respectively. After limiting to arange of 0 to 100%, the values in box 116 and ring 120 becomeindistinguishable at values of 1.0000 and 1.0000.

These color matrix transform encode/decode mismatch patterns aredetectable by visual inspection, even by someone who cannot easilydistinguish colors. The present invention is not strictly limited todisplaying a shape with a shape, but extends to the more general displayat least two colors that are at least partially adjacent in at least oneof space and time, wherein when a color transformation mismatch occurs,the resulting pattern displays a reduction in contrast between the atleast two colors. For example, the test pattern can include shapes thatare at least partially adjacent that flash in time.

FIG. 8 is a block diagram of an apparatus 124 that can be implemented bythe system 10 of FIG. 2 for generating one “saturated” color channel ofthe above-described pattern of FIGS. 7A and 7B. In the presentembodiment, the other two color channels are both zero. The inputs tothe apparatus 124 are x, y, and t. The x and y inputs indicate thespatial position of a given pixel, and the t input indicates thetemporal position, e.g., frame number. The four parameters are then usedby the value selector 126 to create a brightness value for each pixel.The apparatus 124 also includes a block 128 which selects one of twomultiplication constants 1.0 or c, with which to multiply the x or yinput values.

The value selector 126 selects c if x is greater than X1 and less thanX2, where X2>X1, and y is greater than Y1 and less than Y2, where Y2>Y1.Otherwise value selector 126 selects 1.0. X1, X2, Y1, and Y2 areconstants. This arrangement causes c to be selected in block 130 when apixel is within a rectangular sub-region of a certain region of pixels.The t parameter may be used, for example, to periodically stop selectingc entirely for certain frames, causing the sub-region to appear to“blink”.

The value c may be determined by first determining the increase of gainin a certain color channel caused by encoding with one matrix anddecoding with another. Since this gain is an increase, it will begreater than 1.0. The value of c is then the reciprocal of saidincreased gain.

For example, encoding 100% red with Rec. 601 gives YCbCr values of[0.299, −0.168736, 0.5]. Decoding these with the Rec. 709 matrix givesnot 1.0 as the Rec. 601 matrix would, but 1.08640. The value of c is1/1.08640, or 0.920471. This value is the lowest value that will stillresult in clipping at the decoder, rendering it equal to the surroundingvalue of 1.0, and thus indistinguishable from it.

For the reverse problem (detecting when a Rec.709-encoded signal isdecoded with Rec.601) the red channel does not experience a gainincrease, but the green channel does, so the green channel is selected.Encoding 100% green with Rec. 709 gives YCbCr values of [0.7152,−0.385428, −0.454153]. Decoding these with Rec. 601 gives not 1.00, but1.1722, and the value of c that should be used is its reciprocal:0.85312. This value is the lowest value that will still result inclipping at the decoder, rendering it equal to the surrounding value of1.0, and thus indistinguishable from it.

In another embodiment of the present invention, RGB values can begenerated with luma and chroma parts, the luma part of which remainsunchanged unchanged after having been transcoded from one domain toanother, i.e., following a transcoding operation between two specificY′CbCr colorspaces. An iso-luminant test pattern can be generated fortransformations between the Rec.601 standard and the Rec.709 standardfrom the results of the following analysis. The present invention is notstrictly limited to the Rec. 601 and Rec. 709 standards or R-G-B colorspaces, and the analysis can be applied in a similar fashion to othersvideo transmission standards and color spaces (e.g.,yellow-cyan-magneta) as would be understood by one skilled in the art.

Given Y and one of the R,G,B values, the other two of the RGB values aredetermined such thatY=(R601*R)+(G601*G)+(B601*B) and Y=(R709*R)+(G709*G)+(B709*B)where R601, G601, B601 are constant weights between 0 and 1 based on theRec. 601 standard, respectively, R709, G709, and B709 are constantweights between 0 and 1 based on the Rec. 709 standard, respectively,and R, G, and B are red, green, and blue variables whose range islimited to 0 to 1.

For a given value of Y, each equation represents a plane in RGB space.For the two standards in question (Rec.601 and Rec.709), these planesintersect, forming a line in RGB space. Points on this line, that is,RGB triplets corresponding to points on this line, have the same lumavalue in both standards.

There are three variables (RGB) but only two equations. One of thevariables needs to be chosen along with the chosen Y value, and then theother two variables may be derived. The choice for both Y and one of theRGB values should lie between 0 and 1, but some combinations can resultin the other two variables falling outside the range 0 to 1. (The rangemay be scaled to 0 to N, such as N=255, or M to N, where M is 16 and Nis 235, for example.) Because the blue coefficients (B601 and B709) areboth smaller than any of the others, the B variable has very littleinfluence over the final Y value. Choosing a large value of G and a lowvalue for Y, for instance, would result in a negative value for B, whichis not valid.

For convenience invalid values can be avoided by choosing B and Y andthen deriving R and G. For Y=0.5 and B varying between 0 and 1, theresulting R and G values will always be valid. Other means to assurevalid values will occur to those of ordinary skill in the art.

Table 1 is a table of values demonstrating 11 values of blue and thecorresponding values of red and green that result in the luma values of0.5 for both Rec.601 and Rec.709. When the blue signal is a zone plateand Y is set to 0.5, the RGB image that results is shown in FIG. 9, withY stripes having the reference number 132. Measurements may be madeusing the test pattern with a confidence that the luma values areindependent of whether the pattern has been transcoded to anothercolorspace.

TABLE 1 Red Green Blue Y601 Y709 0.719832 0.485128 0.000000 0.5000000.500000 0.675865 0.488103 0.100000 0.500000 0.500000 0.631899 0.4910770.200000 0.500000 0.500000 0.587933 0.494051 0.300000 0.500000 0.5000000.543966 0.497026 0.400000 0.500000 0.500000 0.500000 0.500000 0.5000000.500000 0.500000 0.456034 0.502974 0.600000 0.500000 0.500000 0.4120670.505949 0.700000 0.500000 0.500000 0.368101 0.508923 0.800000 0.5000000.500000 0.324135 0.511897 0.900000 0.500000 0.500000 0.280168 0.5148721.000000 0.500000 0.500000

FIG. 10 is a block diagram of an apparatus 134 that can be implementedin by the system 10 of FIG. 2 for generating an iso-luminant testpattern for transformations between the Rec.601 standard and the Rec.709standards, constructed in accordance with an embodiment of the presentinvention. FIG. 10 shows a processing block 136 that processes a valueof blue and a value of the desired luma and produces corresponding redand green values. These red, green, and the original blue value, whensummed according to the luma coefficients of either the ITU-601 orITU-709, produces the same luma value as presented to the processingblock 136.

The above process may be implemented in two steps. The first stepinvolves calculating the red value from the blue and luma values, andthe second step involves calculating the green value from the blue andluma values. The second step can use the result of the first step as ashortcut. An implementation in C code for the first step is as follows:

float isoLumaRed( const float blue, const float luma ) { const floatr709 = .2126; const float b709 = .0722; const float g709 = .7 152; constfloat r601 = .299; const float b601 = . 114; const float g60 1 = .587;return (luma*(l /g709 −l/g601 ) − ((b709/g709)−(b601 /g601))*blue) I(r709/g709 − r601/g601); }An implementation in C code for the second step is as follows:

float isoLumaGreen( const float blue, const float luma ) { const floatr709 = .2126. cons t float b709 = .0722; const float g709 = .7152;return −((r709/g709)* isoLumaRed( blue, luma) + (b709/g709)*blue −luma*(1/g709)); }Note that in this embodiment the second step (isoLumaGreen) incorporatesthe first step (isoLumaRed).

According to another embodiment of the present invention, lipsync errorcan be detected and quantified as a video and audio test sequence thatincludes a plurality of visual indicators. FIG. 11 shows an examplecomposite test pattern 138 comprising a video frame sequence forindicating lipsync error, constructed in accordance with an embodimentof the present invention. The video frame sequence has two or moredistinct visual features that vary in time, each of the featuresappearing in either a primary form or in one of one or more secondaryforms in any given frame, and a corresponding audio sequence containingone or more changes in the sound that correspond to a change in form ofone or more of the video features. The relevant portion of the compositetest pattern 138 is a horizontal row of gray tick marks 140 (i.e., the“primary form”), corresponding to each frame being currently viewed in atwo-second period, just above the color bar pattern 142. The firstframe, frame 0, has its bright tick mark 144 at the center of the row.Note that the center tick mark 144 is brighter (i.e., the “secondaryform”) and larger than the other tick marks. The “bright” tick mark 144“moves” from center to the right as shown in a series of framesillustrated in FIG. 12. The “bright” tick mark 144 of FIG. 11 and theframes A, B, C and D in FIG. 12 reverses and moves from the right sideto the far left side, reverses and moves toward the center again so asto give the appearance of smooth motion. The final frame's bright markis one mark to the left of center, so that when the sequence is looped,motion appears to be continuous.

The corresponding audio sequence contains a “tick” sound in both leftand right channels at the time of frame 0. (It has a different soundingtick in the right channel when the moving tick reaches the right side,and a similar tick in the left channel when the tick reaches the leftside. These left and right ticks allow detection of a left-right channelreversal error).

To detect lipsync offset, a viewer focuses on a flashing video tick markand listens for the audio tick. The viewer continues to observe thebright video tick mark and listens to the audio tick until a decision ismade as to whether the audio tick precedes or follows the flash of thevideo tick mark (or if it is in sync). If the video and audio tick marksare not in sync, another video tick mark is chosen that appears to bemore closely in sync. This process is repeated until the viewer findsthe closest video tick mark that flashes when the audio tick is heard.The number of tick marks from the center tick indicates the number offrames that the audio and video are offset from each other.

Alternative forms for indicating a difference between primary andsecondary forms of the video indicators of lipsync error include adifference in brightness and/or color, size, and/or shape or differentgeometric shapes such as rectangles, triangles, or circles. The distinctvideo features for indicating lipsync error can be arranged in a patternsuch as a linear pattern. The form changes of spatially-adjacentfeatures can be temporally adjacent.

FIG. 13 is a block diagram of an apparatus 146 that can be implementedwith the system 10 of FIG. 2 for generating a video/audio “test pattern”for detecting and evaluating lipsync error, constructed in accordancewith an embodiment of the present invention. The apparatus 146 createssynchronized audio and video sequences. The apparatus 146 includes aframe counter 148, a video sequence generator 150, and an audio sequencegenerator 152. The input frame pulse occurs once per video frame, andthe frame counter 148 counts the frame pulses and outputs a value thatincreases by one for each frame pulse. A frame number corresponding tothe value of the frame counter 148 is fed to the video sequencegenerator 150. The video sequence generator 150 creates a video imagesequence according to the code listed hereinbelow and triggers the audiosequence generator 152 to start or modify its output at the video framecorresponding to an “in sync” condition.

The following computer code describes the process by which the lipsyncvideo data are created according to the apparatus of FIG. 13.

forEachNew( frameNum ) /* get the new frame number from the input */ {frameRate = 30; /* frames per second */ background = 96; fullscale =255; ticWidth = 3; /* create the output image array and fill with gray*/ CreateArray( tmp, width, height ); fill( tmp, background ); /* createa dim tic. This will be copied to several places in the output. */CreateArray( tic, ticWidth, height/2 ); fill( tic, fullscale/2 ); /*place static, dim tick marks, one for each frame in one second */ for(ii = 0; ii < frameRate; ii++ )  { position = width / 2 * cos( PI * ii /frameRate ); position += width/2; /* ‘zero’ is in the middle of thearray */ position −= ticWidth/2; /* make it centered on the positionhorizontally */ paste( tic, tmp, position, height / 4 ); /* if this isthe center tic, make it taller */ if ( ii == frameRate/2 ) { paste( tic,tmp, position, 0 ); /* more on top */ paste( tic, tmp, position, HEIGHT(tic ) ); /* more on bottom */ }  } /* make and place a single brightpointer tick. This is the one that will appear to move. */ CreateArray(tic, ticWidth, height/2 ); fill( tic, fullscale ); position = width /2 * sin( 1 * PI * frameNum / frameRate ); position += width/2; /* (sameas code above, q.v.) */ position −= ticWidth/2; paste( tic, tmp,position, height / 4 ); /* place the bright tic */ /* if this is thecenter tic, make it taller. Also, trigger the audio click! */ if (frameNum % frameRate == 0 ) { paste( tic, tmp, position, 0 ); /* add totop */ paste( tic, tmp, position, HEIGHT( tic ) ); /* add to bottom */ /* cause the audio generator to start playing or modify its outputevery other second, as bright tic moves left to right (but not as itmoves right to left). */ if ( frameNum % ( 2 * frameRate ) == 0 )sendAudioTriggerPulse( ); } outputThisFrame( tmp ); }

It is to be understood that the exemplary embodiments are merelyillustrative of the invention and that many variations of theabove-described embodiments may be devised by one skilled in the artwithout departing from the scope of the invention. It is thereforeintended that all such variations be included within the scope of thefollowing claims and their equivalents.

What is claimed is:
 1. A method for detecting video color transformationmismatches in video equipment, comprising: generating a test pattern,the test pattern comprising at least two colors; processing the testpattern through video equipment to obtain a color transformed testpattern; and displaying the color transformed test pattern on a displayto a viewer, wherein when a color transformation mismatch occurs due toprocessing with incompatible video transmission standards, the displayedcolor transformed test pattern exhibits a reduction in contrast betweenthe at least two colors.
 2. The method of claim 1, wherein one of the atleast two colors is clipped in at least one of the video equipment andthe display.
 3. The method of claim 1, wherein, when a colortransformation mismatch occurs, the at least two colors becomesubstantially indistinguishable after passing though one of the videoequipment and the display.
 4. The method of claim 1, wherein when onecolor of the at least two colors is 100% of its range and another colorof the at least two colors is set to the reciprocal of the gain increaseto that color that occurs when a transformation mismatch is present inthe video equipment.
 5. The method of claim 1, wherein the colormismatch transformation occurs between equipment being encoded ordecoded with the Rec. 709 and Rec. 601 standards.
 6. The method of claim1, wherein the at least two colors are at least partially adjacent in atleast one of space and time and the at least two colors of the testpattern are represented as a shape within a shape.
 7. The method ofclaim 1, wherein the at least two colors are at least partially adjacentin at least one of space and time and the at least two colors of thetest pattern are represented as adjacent shapes that flash in time.
 8. Asystem for detecting video color transformation mismatches in videoequipment, comprising: a processor being configured for: generating atest pattern, the test pattern comprising at least two colors;processing the test pattern through video equipment to obtain a colortransformed test pattern; and displaying the color transformed testpattern on a display to a viewer, wherein when a color transformationmismatch occurs due to processing with incompatible video transmissionstandards, the displayed color transformed test pattern exhibits areduction in contrast between the at least two colors.
 9. The system ofclaim 8, wherein one of the at least two colors is clipped in at leastone of the video equipment and the display.
 10. The system of claim 8,wherein, when a color transformation mismatch occurs, the at least twocolors become substantially indistinguishable after passing though oneof the video equipment and the display.
 11. The system of claim 8,wherein when one color of the at least two colors is 100% of its rangeand another color of the at least two colors is set to the reciprocal ofthe gain increase to that color that occurs when a transformationmismatch is present in the video equipment.
 12. The system of claim 8,wherein the color mismatch transformation occurs between equipment beingencoded or decoded with the Rec. 709 and Rec. 601 standards.
 13. Thesystem of claim 8, wherein the at least two colors are at leastpartially adjacent in at least one of space and time and the at leasttwo colors of the test pattern are represented as a shape within ashape.
 14. The system of claim 8, wherein the at least two colors are atleast partially adjacent in at least one of space and time and the atleast two colors of the test pattern are represented as adjacent shapesthat flash in time.
 15. A non-transitory computer-readable mediumcarrying one or more sequences of instruction for detecting video colortransformation mismatches in video equipment, wherein execution of theone of more sequences of instructions by one or more processors causesthe one or more processors to perform the step of: generating a testpattern comprising at least two colors, such that when the test patternis color transformed by equipment and the color transformed test patternis displayed on a display, a reduction in contrast between the at leasttwo colors of the displayed color transformed test pattern indicatesthat the color transformation of the test pattern was performed by theequipment with incompatible video transmission standards.
 16. Thecomputer-readable medium of claim 15, wherein one of the at least twocolors is clipped in at least one of the video equipment and thedisplay.
 17. The computer-readable medium of claim 15, wherein, when acolor transformation mismatch occurs, the at least two colors becomesubstantially indistinguishable after passing though one of the videoequipment and the display.
 18. The computer-readable medium of claim 15,wherein when one color of the at least two colors is 100% of its rangeand another color of the at least two colors is set to the reciprocal ofthe gain increase to that color that occurs when a transformationmismatch is present in the video equipment.
 19. The computer-readablemedium of claim 15, wherein the color mismatch transformation occursbetween equipment being encoded or decoded with the Rec. 709 and Rec.601 standards.
 20. The computer-readable medium of claim 15, wherein theat least two colors are at least partially adjacent in at least one ofspace and time and the at least two colors of the test pattern arerepresented as a shape within a shape.